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OSD-Level  Acquisition  Policy  and  Process  Regimes, 

1 970-2009 


1 .  The  Defense  Systems  Acquisition  Review  Council 
(DSARC),  1970-1982 

2.  The  Post-Carlucci  Initiatives  DSARC,  1983-1989 

3.  The  Defense  Acquisition  Board  (DAB),  1990-1993 

4.  Acquisition  Reform  (AR),  1994-2000 

5.  The  DAB  -  Post  Acquisition  Reform,  2001-2009 


Useful  reference:  J.  Ronald  Fox,  Defense  Acquisition  Reform,  1960  to  2009:  An 
Elusive  Goal  (Washington,  DC:  U.S.  Army  Center  of  Military  History,  2011). 
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The  Outcome  Metric — Growth  in  Program  Acquisition 

Unit  Cost  (PAUC) 


■  The  research  used  PAUC  growth  for  151  Major  Defense  Acquisition 
Programs  (MDAPs)  that  passed  MS  ll/B  during  FY  1970-FY  2007. 

■  Each  of  these  MDAPs  went  into  production 

■  There  are  no  cancelled  programs  in  the  sample. 


■  PAUC  growth  is  measured  from  the  MS  ll/B  baseline  and 

normalized  to  the  MS  ll/B  total  quantity  acquired. 

■  PAUC  growth  over  the  entire  acquisition  cycle  is  associated  with 
the  Fiscal  Year  in  which  the  MDAP  passed  MS  ll/B;  for  example: 

■  PAUC  growth  for  the  F-22  over  FY  1991-FY  2006  is  assigned  to  FY  1991,  the 
year  in  which  the  F-22  passed  MS  II. 

■  The  average  PAUC  growth  for  FY  1987-FY  1993  is  the  average  quantity 
normalized  PAUC  growth  of  all  MDAPs  that  passed  MS  II  during  those  years. 
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Average  PAUC  Growth  by  Acquisition  Regimes 


Acquisition  Regime 

Average 

PAUC 

Growth 

DSARC  (1970-1982) 

32% 

Post  Carlucci  Initiatives 

DSARC  (1983-1989) 

19% 

DAB  (1990-1993) 

36% 

Acquisition  Reform 
(AR)  (1994-2000) 

66% 

DAB  post  AR  (2001-2007) 

19% 

There  is  a 
paper  here 
only 

if  I  get  very 
lucky. 
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_ _ _ 

There  is  no  evident  trend  in  PAUC  Growth  across  acquisition  regimes. 


IDA 


Eureka!  Average  PAUC  Growth  by  Funding  Climate 


Relatively  Constrained 

Relatively  Accommodating 

Funding  Climate 

Funding  Climate 

1970-1980 

35% 

1981-1986 

12% 

1987-2002 

53% 

2003-2007 

7% 

Average  PAUC  growth  is  much  higher  for  MDAPs  that  passed  MS  II/B  during 
bust  periods  than  those  that  passed  during  boom  periods. 
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Methodology 


Acquisition  Regime 


Defense  Systems 
Acquisition  Review 
Council  (DSARC) 


Post  Carlucci  Initiatives 
DSARC 


Defense  Acquisition  Board 
(DAB) 


Acquisition  Reform  (AR) 


DAB  postAR 


We  have  eight  natural  experiments  on  the  effect  of  acquisition 
regime  and  three  on  the  effect  of  funding  climate. 
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Average  PAUC  Growth  by  Acquisition  Regime  and 

Funding  Climate 


Acquisition  Regime 

Relatively  C 
Funding  Envi 

Constrained 
ronment  (FY) 

Defense  Systems 
Acquisition  Review 

Council  (DSARC) 

1970-1980 

35%  (42) 

Post  Carlucci  Initiatives 
DSARC 

1987-1989 

34%  (11) 

Defense  Acquisition  Board 
(DAB) 

1990-1993 

36%  (11) 

Acquisition  Reform  (AR) 

1994-2000 

66%  (27) 

DAB  postAR 

2001-2002 

57%  (6) 

Relatively  Accommodating 


1981-1982 


1983-1986 


11%  (6) 


13%  (29) 


No  observations 


2003-2007 


7%  (19) 


Loosely,  acquisition  regime  does  not  matter  for  PAUC  growth  but  funding 
climate  does. 
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Statistical  Analysis  and  Related  Issues 


■  An  Analysis  of  Variance  (ANOVA)  was  done  for  each  budget 
climate.  For  neither  climate  were  any  of  the  differences  across 
regimes  in  average  PAUC  growth  statistically  significant. 

■  A  standard  t-test  was  used  to  test  the  differences  in  average  PAUC 
growth  between  funding  regimes.  Average  PAUC  growth  in  the  bust 
period  was  significantly  higher  (P<  0.01)  in  each  of  the  three  cases 
for  which  the  comparison  could  be  made. 

■  There  is  a  distinct  bust-boom-bust-boom  pattern  in  funding  climate 
over  1970-2007.  If  budget  climate  is  a  proxy  for  some  other 
variable(s)  those  variables  would  need  to  have  the  same  pattern. 
There  are  no  obvious  candidates. 

■  The  paper  (Appendix  B)  provides  evidence  that  PAUC  growth  is  not 
systematically  influenced  by  changes  in  budget  climate  post  MS 
ll/B.  Further  analysis  to  evaluate  this  far  reaching  conclusion  is 
underway. 
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Additional  Results 


■  40  of  the  151  MDAPs  in  the  sample  had  a  PAUC  growth  of  >  50%. 
Not  weighted  by  program  size,  these  account  for  just  over  three- 
quarters  of  cost  growth. 

■  36  of  these  40  programs  passed  MS  ll/B  in  bust  periods.  Nearly  40  percent  of  all 
programs  that  passed  MS  ll/B  in  bust  periods  had  a  PAUC  growth  of  at  least  50 
percent. 

■  Only  4  programs  with  PAUC  growth  of  at  least  50  percent  passed  MS  ll/B  in  a 
boom  period. 

■  29  of  the  1 51  MDAPs  for  which  the  study  had  a  PAUC  growth 
estimate  showed  negative  cost  growth. 

■  Nearly  one-third  of  the  MDAPs  that  passed  MS  ll/B  in  a  boom  funding  climate 
had  negative  PAUC  growth. 

■  The  frequency  of  negative  PAUC  growth  was  much  lower  (about  12  percent)  in 
programs  that  passed  MS  ll/B  in  bust  periods. 

The  PAUC  growth  problem  is  not  systemic;  it  is  largely  one  of  programs 
that  passed  MS  II/B  in  bust  funding  climates  and  had  exceptionally  high 
PAUC  growth. 
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Implications  for  Debates  on  Acquisition  Reform 


■  It  seems  unlikely  that  further  changes  in  the  acquisition 
process  will  have  a  major  effect  on  PAUC  growth. 

■  The  relevant  context  for  understanding  very  high  PAUC 
growth  is  the  interface  between  the  acquisition  process 
and  the  program/budget  process. 

■  The  underlying  cause  of  persistent  high  PAUC  growth  is 
not  a  deeply  established  culture  of  the  DoD  acquisition 
organizations  and  their  professional  employees. 
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Briefing  is  drawn  from  IDA  Paper  P-5126 


IDA  Paper  P-5126,  “Evidence  on  the  Effect  of  DoD  Acquisition 
Policy  and  Process  on  Cost  Growth  of  Major  Defense 
Acquisition  Programs”  is  available  at: 

https://www.ida.Org/~/media/Corporate/Files/Publications/IDA  Do 

cuments/CARD/P-51 26.ashx 


To  obtain  a  hard  copy,  contact  IDA  Library  Reference  Services: 

■  Phone:  (703)  845-2087 

■  E-mail:  refdesk@ida.org 


The  hard  copy  of  the  paper  comes  with  a  CD  containing  the  data  used  and 
spreadsheets  used  in  the  PAUC  growth  computations. 
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Backup 
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Number  of  MDAPs  with  PAUC  Growth  >  50  Percent 


Acquisition  Regime 

Topline  F 
Constrai 

Relatively 
ned  (FY) 

DSARC 

1970-1980 

15  of  42 

Post  Carlucci  DSARC 

1987-1989 

2  of  11 

DAB 

1990-1993 

3  of  11 

AR 

1994-2000 

13  of  27 

DAB  post  AR 

2001-2002 

3  of  6 

Total 

36  of  97  (37%) 

Topline  Relatively 
Accommodating  (FY) 


1981-1982 


1983-1986 


Oof  6 
3  of  29 


No  observations 


2003-2007 


1  of  19 


4  of  54  (7%) 


The  probability  of  a  PAUC  growth  >  50%  was  27  percentage  points 
higher  in  a  Relatively  Constrained  funding  climate  than  in  a  Relatively 

Accommodating  funding  climate. 
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IDA  Percent  of  PAUC  Growth  Accounted  for  by  Observations  >  50% 


Acquisition  Regime 

Relatively  Constrained 
Funding  Climate  (FY) 

DSARC 

1970-1980 

73% 

Post  Carlucci  DSARC 

1987-1989 

62% 

DAB 

1990-1993 

64% 

AR 

1994-2000 

90% 

DAB  post  AR 

2001-2002 

75% 

Relatively  Accommodating 
Funding  Climate  (FY) 


1981-1982 


1983-1986 


70% 


No  observations 


2003-2007 


43% 


Not  weighted  by  program  size,  MDAPs  with  a  PAUC  growth  of  >  50% 
account  for  just  over  three-quarters  of  quantity  adjusted  cost  growth. 
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Number  of  MDAPs  with  Negative  PAUC  Growth 


Acquisition  Regime 

Relatively 
Funding  ( 

Constrained 
Climate  (FY) 

DSARC 

1970-1980 

5  of  42 

Post  Carlucci  DSARC 

1987-1989 

2  of  11 

DAB 

1990-1993 

1  of  11 

AR 

1994-2000 

4  of  27 

DAB  post  AR 

2001-2002 

Oof  6 

Total 

12  of  97  (12%) 

Relatively  Accommodating 
Funding  Climate  (FY) 


1981-1982 


1983-1986 


2  of  6 


10  of  29 


No  observations 


2003-2007 


5  of  19 


17  of  54  (31%) 


The  probability  of  negative  PAUC  growth  is  21  percentage  points  lower  for 
MDAPs  that  passed  MS  ll/B  in  a  Relatively  Constrained  funding  climate. 
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Alternative  Explanations  for  Negative  PAUC  Growth  and 

PAUC  Growth  >  50  Percent 


■  P-5126  concludes  that  “...the  relevant  context  for  understanding  PAUC 
growth  is  the  interface  between  the  acquisition  process  and  the  resource 
allocation  process.” 

■  This  conclusion  includes  two  possibilities: 

-  Major  acquisition  programs  more  frequently  adopt  more  optimistic  costing  and 
programmatic  assumptions  in  Relatively  Constrained  funding  climates  compared 
to  Relatively  Accommodating  climates. 

-  In  Relatively  Constrained  funding  climates,  some  MDAPs  are  issued  top-down 
guidance  (possibly  from  the  Program/Budget  process)  that  results  in  more 
optimistic  costing  and  programmatic  assumptions.  In  Accommodating  funding 
climates,  some  MDAPs  are  over-funded  so  that  they  can  be  used  as  a  “bank”  to 
hold  funds  for  later  commitment. 

■  Each  explanation  implies  more  negative  PAUC  growth  in  Accommodating 
climates  and  more  MDAPs  with  PAUC  growth  of  >  50%  in  Relatively 
Constrained  climates. 

■  Each  of  the  two  explanations  probably  is  correct  in  some  instances. 

■  P-5126  did  not  attempt  to  distinguish  between  the  two  statistically. 


15 


IDA 


Bias  in  the  MS  ll/B  PAUC  Estimate  and  the  Frequency  of  PAUC 
Growth  >  50  Percent  and  Negative  PAUC  Growth 


Relatively  Accommodating 
Funding  Climate 


Relatively  Constrained 
Funding  Climate 
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Average  PAUC  Growth  Excluding  Negative  Observations 

and  Observations  >  50  Percent 


Acquisition  Regime 

Relatively  Constrained 
Funding  Climate  (FY) 

DSARC 

1970-1980 

21%  (22) 

Post  Carlucci  DSARC 

1987-1989 

22%  (7) 

DAB 

1990-1993 

22%  (7) 

AR 

1994-2000 

22%  (10) 

DAB  post  AR 

2001-2002 

29%  (3) 

Funding  Climate  (FY) 


1981-1982 


1983-1986 


22%  (4) 


14%  (16) 


No  observations 


2003-2007 


10%  (13) 


Note:  Numbers  in  parentheses  are  the  number  of  observations  available. 


■  Some  decrease  in  variation  is  guaranteed  by  exclusion  of  the  negative 
observations  and  those  of  >  50%.  Even  so,  there  is  a  remarkable 
consistency  across  the  cells  of  PAUC  growth. 

■  ANOVA  did  not  find  statistically  significant  difference  in  PAUC  growth  across 
the  acquisition  regimes  for  either  funding  climate. 

■  PAUC  growth  was  significantly  lower  in  the  Accommodating  climate  for  the 
DAB  post  AR  regime. 
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(1 )  Acquisition  Regime  in  Periods  of  Relatively 
Constrained  Procurement  Funding  Climate 


Acquisition  Regime 

Relatively  Coi 

istrained  (FY) 

DSARC 

1970-1980 

35%  (42) 

Post  Carlucci  DSARC 

1987-1989 

34%  (11) 

DAB 

1990-1993 

36%  (11) 

AR 

1994-2000 

66%  (27) 

DAB  post  AR 

2001-2002 

57%  (6) 

Note:  Number  of  observations  available  in  parentheses. 


The  differences  in  average  PAUC  growth  across  acquisition  regimes  also  are 
not  statistically  significant  for  the  Relatively  Constrained  funding  climate. 
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(2)  Acquisition  Regime  in  Periods  of  Relatively 
Accommodating  Procurement  Funding  Climate 


Acquisition  Regime 

Relatively  Acco 

mmodating  (FY) 

DSARC 

1981-1982 

11%  (6) 

Post  Carlucci  DSARC 

1983-1986 

13%  (29) 

DAB 

A  f  _  _  f _ 

AR 

/vo  oDservaiions 

DAB  post  AR 

2003-2007 

7%  (19) 

Note:  Number  of  observations  available  in  parentheses. 


The  differences  in  average  PAUC  growth  across  acquisition  regimes  are  not 
statistically  significant  for  the  Relatively  Accommodating  funding  climate. 
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(3)  PAUC  Growth  and  Procurement  Funding  Climate  Given 

Acquisition  Regime 


Acquisition  Regime 

Relatively  Constrained  (FY) 

Relatively  Accor 

nmodating  (FY) 

DSARC 

1970-1980 

1981-1982 

11%  (6) 

Post  Carlucci  DSARC 

1987-1989 

1983-1986 

13%  (29) 

DAB 

No  observations 

AR 

DAB  post  AR 

2001-2002 

57%  (6) 

2003-2007 

7%  (19) 

Note:  Number  of  observations  available  in  parentheses. 


Average  PAUC  growth  was  significantly  less  in  the  Relatively 
Accommodating  funding  climate  for  each  of  the  three  acquisition 
regimes  for  which  the  comparison  can  be  made. 
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Selection  of  Break  Points  between  Funding  Climates 


■  This  research  used  only  two  funding  climate  categories — Relatively 
Constrained  and  Relatively  Accommodating. 

■  Our  touchstone  in  selection  of  break  points  was  major  shifts  in  the 
expectation  about  future  funding  of  senior  DoD  decision  makers. 

■  We  used  three  events  to  identify  the  break  points  between  funding  climates: 

-  The  invasion  of  Afghanistan  by  the  USSR  in  late  December  1979. 

-  The  passage  of  the  Gramm-Rudman-Hollings  Act  in  December  1985;  and 

-  The  terrorist  attack  on  the  U.S.  on  Sept.  1 1 , 2001 . 

■  Senior  decision  makers  could  reasonably  expect  each  of  these  events  to 
result  in  major  and  sustained  changes  in  the  defense  funding  climate. 

■  After  examining  contemporary  policy  statements  and  events  we  selected: 

-  FY  1981  as  the  first  year  of  the  Carter-Reagan  buildup; 

-  FY  1986  as  the  final  year  of  the  Carter-Reagan  buildup;  and 

-  FY  2003  as  the  first  year  of  the  post-9/1 1  defense  buildup. 
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|rt .  PAUC  Growth  Acquisition  Regime  and  Procurement  Funding 

Climate — Alternative  Break  Points 


Note:  Number  of  observations  available  in  parentheses. 


Two  conclusions  change  if  the  alternative  break  points  are  used: 

1.  Average  PAUC  growth  for  the  Acquisition  Reform  years  is  significantly 
higher  than  in  other  acquisition  regimes. 

2.  The  difference  in  PAUC  growth  between  the  two  funding  climates  is  not 
statistically  significant  for  the  Post  Carlucci  DSARC  acquisition  regime. 
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